Learning Preconditions for Control Policies in Reinforcement Learning

This paper describes a method which senses changing environment by collecting failed instances, uses concept learning for acquiring a precondition for a control policy, and modifies the policy partially in reinforcement learning. The precondition of a policy represents the condition for reaching goals using the policy. Our method learns the precondition of a policy from the instances of policy success or failure by concept learning like learning the preconditions of an action model from the instances of action success or failure by concept learning. Concept learning which generalizes experienced states provides an ability to modify its behavior in inexperienced states. We experimented our method using a reinforcement learning method profit sharing and a decision tree learning system C4.5. Our method adapted to a changing environment faster than re-learning and continuing reinforcement learning. Besides, we confirmed that concept learning provided a method to adapt effectively. Since our method does not restrict reinforcement learning, it is applicable to domains where reinforcement learning is applicable. It is easy to extend our system to use repeatedly for subsequent changes of environment.